Towards basic emotion recognition using players body and hands pose in virtual reality narrative experiences

Currently, players position recognition in most Virtual Reality applications is limited to the evident usage, like translating players avatar within the virtual environment, or using the view point at head height, without considering the posture at any time. In this article, we propose studying the player’s body and hand expression, not only to recognize obvious interaction patterns but poses that, even without conscience, transmit information about the basic emotions of such player. That way, each time it is played, the result is altered without a conscious effort, the experience of interactive narrative resulting of computing generation depends on the input signals, which adds a layer of depth and enriches system decision making and conversation with non-player characters, for example. Our proposal is based on a system that relies on a system with a neural network which can recognize poses, according to the specialists, associated to basic human mood. After a simple calibration and a reasonable training, this system can be used, without the need of additional accessories, with the main Virtual Reality devices existing today. This article also discusses new paths of research and applications that arise around this system, many in the field of computer entertainment, but also in other areas such as therapy for patients with emotional and social communication problems and disorders.